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BACKGROUND OF THE INVENTION 

1. FIELD OF THE INVENTION: 

The present invention relates to an omniazimuthal 
visual system for use in a visual sensor, such as 
surveillance cameras, etc. , which has a wide angle of view 
that enables omniazimuthal observation. 

2. DESCRIPTION OF THE RELATED ART: 

In recent years, in the field of visual sensors 
such as surveillance cameras, etc. , in various practical 
applications, a combination of a camera and a computer 
executes tasks which would have been carried out by a human 
with his/her own eyes in conventional applications. In 
general, a camera used in such applications has a limited 
angle of view. In some applications, a wide-angle lens 
(e.g. , fisheye lens) is used for the purpose of obtaining 
a wider angle of view. Furthermore, in the field of mobile 
robots, studies for practical use of a mirror having a 
shape of a surface of revolution, such as a conical mirror, 
a spherical mirror, a hyperboloidal mirror, etc., are 
energetically carried out for the purpose of obtaining 
a wide angle of view. 
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In conventional visual sensors, an omniazimuthal 
image obtained by an optical system and an imaging device 
is processed by software used by a computer so as to be 
transformed into an image which is easier for a human to 
see, such as a panoramic image or the like. 

Figure 10 shows a conventional omniazimuthal 
visual system 1000. The omniazimuthal visual 

system 1000 includes: an optical system 1001 which 
utilizes a wide angle lens (e.g. , f isheye lens ) or a mirror 
having a shape of a surface of revolution ( a conical mirror , 
a spherical mirror, a hyperboloidal mirror, etc.); an 
imaging section 1002 for converting an optical image 
obtained by the optical system 1001 into image data; and 
a computer (workstation) 1007 including an image 
converter which performs software processing, a display, 
and a display controller. In this omniazimuthal visual 
system 1000, a round- shape optical image obtained by the 
optical system 1001 is converted by the imaging 
section 1002 into image data of the round- shape image. 
This image data is supplied to the computer 1007, and 
processed by the computer 1007 using software into an 
image which is easier for a human to see, such as a square 
panoramic image or a perspective image. 
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The image transformation processing, through 
which a round- shape optical image is transformed into an 
image which is easier for a human to see, such as a square 
panoramic image or a perspective image, requires a number 
of coordinate transformations by which a polar coordinate 
is transformed with a trigonometric function into a 
rectangular coordinate. Furthermore, the coordinate 
transformation requires high accuracy. 

Accordingly, in the conventional omniazimuthal 
visual system 1000, when the image transformation 
processing is carried out by software used by the 
computer 1007, 20 or more steps are required in arithmetic 
processing for a single data transformation even in the 
case of using a hyperboloidal mirror. Therefore, the 
image transformation processing which uses software 
requires a long processing time. 

Among various mirrors having a shape of a surface 
of revolution, a hyperboloidal mirror is an optical 
element which enables a precise central projection 
transformation. With the hyperboloidal mirror, an image 
transformation can be carried out only with linear 
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operations. On the other hand, in the case of using a 
wide-angle lens (e,g., fisheye lens) or another type of 
mirror having a shape of a surface of revolution, such 
as a conical mirror or a spherical mirror, an image 
transformation requires nonlinear operations in addition 
to trigonometric functions. Thus, in the case of using 
the hyperboloidal mirror, a processing speed is 
relatively increased as compared with an optical system 
using a wide angle lens (e.g., fisheye lens) or another 
type of mirror having a shape of a surface of revolution, 
such as a conical mirror or a spherical mirror. 

Furthermore, when software is used in the image 
transformation, even in the case where a result of a 
constant operation is previously stored in a register, 
it is necessary to frequently repeat a step of reading 
out this result from a register, when it is required, and 
temporarily storing a calculation result again in the 
register. Therefore, a data processing time is 
determined according to the number of the repetitions of 
the step. 

Although the processing time required for 
software processing of an image formed by a small number 
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of pixels might be short, a considerably longer processing 
time is required for software processing of an image 
formed by a greater number of pixels. Thus, for an image 
formed by 100,000 pixels, the maximum processing speed 
of the image transformation is several frames per second. 

In the case of a still image, a long software 
processing time does not matter greatly. Therefore, a 
high resolution picture can be obtained even with a 
conventional omniazimuthal visual system. However, in 
the case of a dynamic image, along with an increased number 
of pixels, the processing speed of the software processing 
decreases so that the software processing cannot catch 
up with a required processing speed for dynamic images. 
As a result, the quality of the images obtained is 
significantly deteriorated. Thus, higher processing 
speed is required for processing of a dynamic image. 



SUMMARY OF THE INVENTION 



According to one aspect of the present invention, 
an omniazimuthal visual system includes: an optical 
system capable of obtaining an image of 360^ view field 
area therearound and capable of central projection 
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transformation for the image; an imaging section for 
converting the image obtained through the optical system 
into image data; an image transformation section for 
transforming the image data into display data; a display 
section for displaying a transformed image based on the 
display data from the image transformation section; and 
a display control section for controlling the transformed 
image to be displayed on the display section, wherein the 
image transformation section includes at least one buffer 
memory for temporarily storing the image data and the 
display data, an arithmetic/logic circuit for performing 
coordinate transformation when the image data is 
transformed into the display data, a lookup table of a 
trigonometric function for use in the arithmetic/logic 
circuit, and a CPU for controlling the at least one buffer 
memory, the arithmetic/logic circuit, and the lookup 
table . 

In one embodiment of the present invention, the 
image transformation section includes any of a function 
for transforming the image data into the display data for 
generating a panoramic image, a function for transforming 
the image data into the display data for generating a 
perspective image, a function for determining a 
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transformation region of the image data, a zoom- 
in/zoom-out function, and a pan/ tilt function. 

In another embodiment of the present invention, 
the image transformation section transforms the image 
data which has undergone zoom-in/zoom-out processing by 
the zoom-in/zoom-out function or pan/tilt processing by 
the pan/tilt function into the display data for generating 
a panoramic image or the display data for generating a 
perspective image . 

In still another embodiment of the present 
invention, the arithmetic/logic circuit is formed only 
by linear operation circuits. 

In still another embodiment of the present 
invention, the optical system includes a hyperboloidal 
mirror having a hyperboloidal shape of one sheet of a 
two-sheeted hyperboloid; a rotation axis of the 
hyperboloidal mirror is identical with an optical axis 
of an imaging lens included in the imaging section; and 
a first principal point of the imaging lens is located 
at a focal point of the hyperboloidal mirror. 
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In still another embodiment of the present 
invention, the optical system includes a plurality of 
mirrors each having a shape of a surface of revolution, 
the plurality of mirrors including first and second 
mirrors; and the optical system is structured such that 
the rotation axis of the plurality of mirrors is identical 
with an optical axis of an imaging lens included in the 
imaging section, a first principal point of the imaging 
lens is located at a focal point of the second mirror, 
and light which travels toward a focal point of the first 
mirror is reflected by the first mirror and, thereafter, 
reflected by the second mirror so as to reach the first 
principal point of the imaging lens included in the 
imaging section. 

In still another embodiment of the present 
invention, the imaging section is a digital imaging 
section for transforming the image into digital image 
data . 

In still another embodiment of the present 
invention, the omniazimuthal visual system further 
includes an image recording section for recording an 
output from the imaging section, wherein the image 
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recording section is a digital recording section. 

In still another embodiment of the present 
invention, the CPU includes a parallel operation function 
for controlling the display control section, the image 
transformation section and the display section. 

In the present specification, the phrase "an 
optical system is capable of central projection 
transformation" means that an imaging device is capable 
of acquiring an image which corresponds to an image seen 
from one of a plurality of focal points of an optical 
system. 

Hereinafter, functions of the present invention 
will be described. 

An omniazimuthal visual system according to the 
present invention includes an image transformation 
section having a function for transforming image data 
output from an imaging section into a panoramic image or 
a perspective image, a function for determining a 
transformation region, a zoom-in/zoom-out function, a 
pan/tilt function, etc. The image transformation 
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section includes an input buffer memory for storing image 
data, an arithmetic/logic circuit for image 
transformation processing, a central processing unit 
(CPU) for controlling the image transformation processing, 
a lookup table, and an output buffer memory for storing 
image data which has undergone the image transformation 
processing. Furthermore, the omniazimuthal visual 
system may include an image recording section. 

With the above structure, since the image 
transformation section of the present invention is formed 
by hardware, it is not necessary to store intermediate 
results of the operations in a storage device and to read 
these intermediate results therefrom. Further, it is 
possible to carry out the image transformation processing 
within a delay time between an input and output of the 
hardware for operations (e.g., the image transformation 
section). Thus, the processing speed for image 
transformation processing is faster than that of a 
conventional system which carries out image 
transformation using software. Furthermore, since the 
image transformation section is realized by hardware 
(i.e., it does not include an additional element for 
software processing), the size of the entire 
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omniazimuthal visual system can be reduced. 

In general, a calculation for a trigonometric 
function is carried out by a series expanding method and 
an approximate calculation. The accuracy of the 
calculation is determined according to the number of 
orders of terms to be calculated. Accordingly, when a 
highly accurate calculation is required, the number of 
terms to be calculated is increased. As a result, the 
calculation time considerably increases. According to 
the present invention, a lookup table of a trigonometric 
function is employed, whereby the calculation time can 
be shortened. 

When the zoom-in/zoom-out processing or the 
pan/tilt processing is carried out based on image data 
which is coordinate- transformed for panoramic or 
perspective transformation, it is necessary to provide 
a buffer memory for temporarily storing the 
coordinate- transformed image data. According to the 
present invention, the zoom-in/zoom-out processing or the 
pan/tilt processing is carried out based on input image 
data from an imaging section or an image recording section 
and, thereafter, the panoramic or perspective 
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transformation is carried out . With such an arrangement, 
it is not necessary to provide an additional buffer 
memory . 

According to a first aspect of the present 
invention (embodiment 1), the optical system is formed 
by a hyperboloidal mirror which has a shape of one sheet 
of a two- sheeted hyperboloid. The rotation axis of the 
hyperboloidal mirror is identical with the optical axis 
of an imaging lens included in the imaging section, and 
the first principal point of the imaging lens is located 
at one of focal points (external focal point) of the 
hyperboloidal mirror. In such a structure, an image 
obtained by the imaging section corresponds to an image 
seen from one of the focal points (internal focal point) 
of the hyperboloidal mirror. Therefore, in the panoramic 
or perspective transformation, it is possible to carry 
out coordinate transformation from a polar coordinate to 
a rectangular coordinate only with linear operations 
using the lookup table of a trigonometric function. 

According to a second aspect of the present 
invention (embodiment 2), the optical system includes a 
plurality of convex mirrors having a shape of a surface 
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of revolution. For example, the optical system includes 
two (first and second) convex mirrors having a shape of 
a surface of revolution. The rotation axes of the mirrors 
are identical with the optical axis of an imaging lens 
5 included in the imaging section. The imaging section is 
positioned such that a first principal point of the 
imaging lens is located at a focal point of the second 
mirror. With such a configuration, light which travels 
toward a focal point of the first mirror is reflected by 

10 the first mirror, and light reflected by the first mirror 
is further reflected by an internal face of the second 
mirror so as to reach the first principal point of the 
imaging lens. In the optical system having such a 
structure, an image obtained by the imaging section 

15 corresponds to an image seen from a focal point of the 
first mirror. Therefore, in the panoramic or perspective 
transformation, it is possible to carry out coordinate 
transformation from a polar coordinate to a rectangular 
coordinate only with linear operations using the lookup 

20 table of a trigonometric function . 

When a digital imaging section which digitally 
converts an optical image into digital image data is used 
as the imaging section, data processing can be easily 
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carried out . 

When a digital recording section is used as the 
image recording section, data processing can be easily 
carried out . 

Furthermore, when a CPU including a parallel 
operation function for controlling a display control 
section, an image transformation section and the display 
section, is used, the image transformation processing can 
be carried out at a high speed. 

Thus, the invention described herein makes 
possible the advantage of providing an omniazimuthal 
visual system capable of processing a dynamic image with 
increased number of pixels by means of a high-speed image 
transformation . 

This and other advantages of the present invention 
will become apparent to those skilled in the art upon 
reading and understanding the following detailed 
description with reference to the accompanying figures. 



BRIEF DESCRIPTION OF THE DRAWINGS 



- 15 - 



00R00703 



Figure 1 is a block diagram showing a structure 
of an omniazimuthal visual system 100 according to 
embodiment 1 of the present invention. 

Figure 2 shows a configuration example of an 
optical system according to embodiment 1. 

Figure 3 is a block diagram showing a 
configuration example of an image transformation section 
according to embodiment 1 . 

Figure 4 illustrates an example of panoramic 
(360°) image transformation according to embodiment 1. 
Part (a) shows an input round-shape image. Part (b) 
shows a donut- shape image subjected to the panoramic image 
transformation. Part (a) shows a panoramic image 
obtained by transformation into a rectangular coordinate. 

Figure 5 illustrates a perspective 

transformation according to embodiment 1 . 

Figure 6 shows a configuration example of a 
panoramic transformation logic circuit according to 
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embodiment 1 . 

Figure 7 shows a configuration example of a 
perspective transformation logic circuit according to 
embodiment 1 . 

Figure 8 shows a configuration example of a 
perspective transformation logic circuit according to 
embodiment 1 . 

Figure 9 shows a configuration example of an 
optical system according to embodiment 2. 

Figure 10 is a block diagram showing a 
configuration example of a conventional omniazimuthal 
visual system 1000. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Hereinafter, embodiments of the present invention 
will be described with reference to the drawings. 

( Embodiment 1 ) 

Figure 1 is a block diagram showing a structure 
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of an omniazimuthal visual system 100 according to 
embodiment 1 of the present invention. The 
omniazimuthal visual system 100 includes: an optical 
system 101 capable of obtaining an image of 360° view field 
area therearound and capable of central projection 
transformation for the image; an imaging section 102 for 
converting an optical image obtained by the optical 
system 101 into image data; an image recording 
section 103 for recording the image data received from 
the imaging section 102; an image transformation 
section 104 for transforming the image data received from 
the imaging section 102 . or the image recording 
section 103 into a panoramic image or a perspective image; 
a display section 105 for displaying an output of the 
image transformation section 104; and a display control 
section 106 for selecting an image to be displayed in the 
display section 105 and controlling the size of the image, 
etc . 

Each of these components is now described in 
detail with reference to Figures 1 through 8. 

Figure 2 shows an example of the optical 
system 101 capable of central projection transformation. 
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This optical system uses a hyperboloidal mirror 54 which 
has a shape of one sheet of a two- sheeted hyperboloid, 
which is an example of a mirror having a shape of a surface 
of revolution. The rotation axis of the hyperboloidal 
mirror 54 is identical with the optical axis of an imaging 
lens included in the imaging section 102, and the first 
principal point of the imaging lens which is included in 
the imaging section 102 is located at one of focal points 
of the hyperboloidal mirror 54 (external focal point 
(D) . In such a structure, an image obtained by the imaging 
section 102 corresponds to an image seen from the internal 
focal point ® of the hyperboloidal mirror 54, Such an 
optical system is disclosed in, for example, Japanese 
Laid-open Publication No. 6-295333, and only several 
features of the optical system are herein described. 

In Figure 2, the hyperboloidal mirror 54 is 
formed by providing a mirror on a convex surface of a body 
defined by one of curved surfaces obtained by rotating 
hyperbolic curves around a z-axis (two-sheeted 
hyperboloid) , i.e., a region of the two-sheeted 
hyperboloid where Z>0. This two-sheeted hyperboloid is 
represented as: 
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(X^+yM/a^-ZVb^ = -l 

where a and b are constants for defining a shape of the 
hyperboloid, and c is a constant for defining a focal point 
of the hyperboloid. Hereinafter, the constants a, b, 
and c are generically referred to as "mirror constants". 

The hyperboloidal mirror 54 has two focal points 
® and®- All of light from outside which travels toward 
focal point ® is reflected by the hyperboloidal 
mirror 54 so as to reach focal point <D . The 
hyperboloidal mirror 54 and the imaging section 102 are 
positioned such that the rotation axis of the 
hyperboloidal mirror 54 is identical with the optical 
axis of an imaging lens of the imaging section 102, and 
the first principal point of the imaging lens is located 
at focal point (2). With such a configuration, an image 
obtained by the imaging section 102 corresponds to an 
image seen from focal point ® of the hyperboloidal 
mirror 54, 

Due to such an optical system, when a lookup table 
(LUT) 110 (described later) is used, the image 
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transformation section 104 (Figure 1) can carry out an 
image transformation, i.e., the image transformation 
section 104 can transform image data into display data, 
such as a panoramic image, a perspective image, etc. , only 
with linear operations and without nonlinear operations. 

The imaging section 102 may be a video camera or 
the like. The imaging section 102 converts an optical 
image obtained through the hyperboloidal mirror 54 of 
Figure 2 into digital image data using a solid-state 
imaging device, such as CCD, CMOS, etc. A lens of the 
imaging section 102 may be a commonly- employed spherical 
lens or aspherical lens so long as the first principal 
point of the lens is located at focal point (D. 

The image recording section 103 receives a 
digital output of the imaging section 102 and digitally 
records the digital output (time- series image signal) in 
a tape or a disk- type recording medium such as an MO, an 
MD, a DVD, etc. 

Referring to Figure 3, the image transformation 
section 104 includes an input buffer memory 108, a 
CPU 109, a lookup table (LUT) 110 of a trigonometric 
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function, an arithmetic/logic circuit 111 for image 
transformation, and an output buffer memory 112. These 
components are connected via a bus line 113, In Figure 3, 
two buffer memories, i.e., the input buffer memory 108 
and the output buffer memory 112 are shown. However, 
according to the present invention, it is only necessary 
to provide at least one buffer memory. 

The input buffer memory 108 of the image 
transformation section 104 receives image data from the 
imaging section 102 and the image recording section 103. 
The image data input to the input buffer memory 108 is 
a round- shape image. This round- shape image is 
transferred to the arithmetic/logic circuit 111 and 
subjected to a coordinate transformation from a polar 
coordinate to a rectangular coordinate using the lookup 
table (LUT) 110 of a trigonometric function, whereby the 
round- shape image is transformed into an image which is 
easier for a human to see, such as a square panoramic image 
or a perspective image. The coordinate transformation 
will be described later in detail. Furthermore, the image 
data may be, if necessary, subjected to various image 
transformation processing such as zoom-in/zoom-out, 
pan/tilt, etc., according to alteration of the viewing 
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direction* After the image transformation processing, 
the processed image data is input to the output buffer 
memory 112. A series of steps of the image transformation 
processing are controlled by the CPU 109. 

When image data which has undergone the coordinate 
transformation is subjected to zoom-in/zoom-out 
processing or pan/tilt processing, it is necessary to 
provide a buffer memory for temporarily storing the image 
data which has undergone the coordinate transformation. 
Therefore, in embodiment 1, image data from the imaging 
section 102 or the image recording section 103, which has 
not undergone the coordinate transformation, is used for 
zoom-in/zoom-out processing or pan/tilt processing, and 
image data obtained after the zoom-in/zoom-out processing 
or pan/tilt processing is transformed into a panoramic 
image or a perspective image. With such a procedure, the 
image transformation can be carried out without providing 
an additional buffer memory and, accordingly, an increase 
in size of the system can be prevented. 

The display section 105 may be a monitor, or the 
like, of a cathode-ray tube , LCD, EL (electroluminescence 
device), etc. The display section 105 receives an output 
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from the output buffer memory 112 of the image 
transformation section 104 and displays an image. 

The display control section 106 may be a 
special-purpose microcomputer or the like. The display 
control section 106 selects the type of an image to be 
displayed on the display section 105 (for example, a 
panoramic image, a perspective image, etc., obtained by 
the image transformation in the image transformation 
section 104), and controls the orientation and the size 
of the image . 

A principle of the image transformation by the 
arithmetic/logic circuit 111 is now described. The 
image transformation includes a panoramic transformation 
for obtaining a panoramic (360®) image and a perspective 
transformation for obtaining a perspective image. 
Furthermore, the perspective transformation includes a 
horizontally rotational transfer (horizontal transfer, 
so-called "pan movement") and a vertically rotational 
transfer (vertical transfer, so-called "tilt movement") . 

First, a panoramic (360°) image transformation is 
described with reference to Figure 4, Referring to 
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part (a) of Figure 4, an image 57 is a round-shape image 
input from the imaging section 102 or the image recording 
section 103 to the arithmetic/logic circuit 111 
(Figure 1). Part (b) of Figure 4 shows a donut- shape 
image 58 subjected to the panoramic image transformation. 
Part (o) of Figure 4 shows a panoramic image 59 obtained 
by transforming the image 57 into a rectangular 
coordinate. 

Part (a) of Figure 4 shows the input round- shape 
image 57 which is formatted in a polar coordinate form 
in which the center point of the image 57 is positioned 
at the origin (Xo,Yo) of the coordinates. In this polar 
coordinate, a pixel P in the image 57 is represented as 
P(r,e). Referring to part (o) of Figure 4, in the 
panoramic image 59, a point corresponding to the pixel P 
in the image 57 (part (a) of Figure 4) can be represented 
as P(x,y). When the round-shape image 57 shown in 
part (a) of Figure 4 is transformed into the square 
panoramic image 59 shown in part (o) of Figure 4 using 
a point O(ro,6o) as a reference point, this 
transformation is represented by the following 
expressions: 



- 25 - 



00R00703 



X = 9-eo 
y = r-ro 

When the input round- shape image 57 (part (a) of 
Figure 4) is formatted into a rectangular coordinate such 
that the center point of the round- shape image 57 is 
positioned at the origin of the rectangular coordinate 
system, (Xo,Yo), the point P on the image 57 is 
represented as {X,Y). Accordingly, X and Y are 
represented as: 

X = Xo+rxcos6 
Y = Yo+rxsin6 

Thus , 

0 = arctan{ (Y-Yo)/(X-Xo) }-eo 
r = sqrt{ {X-Xo)^+(Y-Yo)^} 

In the pan movement for a panoramic image, a point 
obtained by increasing or decreasing "8o" of the reference 
point O(ro,6o) by a certain angle A9 according to a 
predetermined key operation is used as a reference point 
for the pan movement. With this reference point for the 
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pan movement, a horizontally panned panoramic image can 
be directly obtained from the input round- shape image 57. 
It should be noted that a tilt movement is not performed 
for a panoramic image. 

Next, a perspective transformation is described 
with reference to Figure 5. In the perspective 
transformation, the position of a point on the input image 
obtained by a light receiving section 2a of the imaging 
section 102 which corresponds to a point in a three- 
dimensional space is calculated, and image information 
at the point on the input image is allocated to a 
corresponding point on a perspective- transformed image, 
whereby coordinate transformation is performed. 

In particular, as shown in Figure 5, a point in 
a three-dimensional space is represented as P(tx,ty,tz) , 
a point corresponding thereto which is on a round- shape 
image (input coordinate system) formed on an imaging lens 
(light receiving section) 2a of the imaging section 102 
is represented as P ( r , 9 ) , the focal distance of the imaging 
lens 2a of the imaging section 102 is F, and mirror 
constants are (a, b, c), which are the same as a, b, and 
c in Figure 2. With these parameters, expression (1) is 
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obtained: 

r = Fxtan( (jr/2)-P) (1) 

In Figure 5, a is an incident angle of light which travels 
from an object point (point P) toward focal point ® with 
respect to a horizontal plane including focal point ®; 
P is an incident angle of light which comes from point P, 
is reflected at point G on the hyperboloidal mirror 54, 
and enters into the imaging section 102 (angle between 
the incident light and a plane perpendicular to an optical 
axis of the imaging lens 2a of the imaging section 102) . 
Algebraic numbers a, p, and B are represented as follows: 

P = arctan( ( (b^+c^)xsin a -2xbxc) / (b^-c^ )xcosa) 
a = arctan(tz/sqrt (tx^+ty^) ) 
9 = arctan( ty/tx) 

From the above, expression (1) is represented as follows: 

r = Fx( ( (b^-c^)xsqrt (tx^ + ty^) )/( (b^+c^) 
xtz-2xbxcxsqrt (tx^ + ty^ + tz^) ) ) 

The coordinate of a point on the round- shape image is 
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transformed into a rectangular coordinate P {X,Y) . X and 
Y are represented as: 

X = rxcosB 

Y = rxsinB 

Accordingly, from the above expressions: 

X = Fx( ( (b^-cMxtx/( (b^+c^)xtz-2xbxc 

xsqrt ( tx^+ty^ + tz^ ) ) ) ' " {2) 

Y = Fx{ ( (b^-c^)xty/( (b^+cMxtz-2xbxc 

xsqrt ( tx^+ty^+tz^ ) ) ) — • ( 3 ) 

With the above expressions, object point 
P (tx,ty,tz) is perspectively transformed onto the 
rectangular coordinate system. 

Now, referring to Figure 5, consider a square 
image plane having width W and height h and located in 
the three-dimensional space at a position corresponding 
to a rotation angle 9 around the Z-axis where R is a 
distance between the plane and focal point ® of the 
hyperboloidal mirror 54, and (() is a depression angle 
(which is equal to the incident angle a) . Parameters of 
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a point at the upper left corner of the square image plane, 
point Q ( txq, tyq, tzq) , are represented as follows: 

txq = (Rcos<t)+(h/2)sin(|))cose-(W/2)sin8 (4) 

tyq = (Rcos())+(h/2)sin(|))sine+(W/2)cos6 (5) 

tzq = Rsin(|)- (h/2 )cos(t) (6) 

By combining expressions (4), (5), and (6) into 
expressions (2) and (3), it is possible to obtain the 
coordinate (X,Y) of a point on the round- shape image 
formed on the imaging lens 2a of the imaging section 102 
which corresponds to point Q of the square image plane. 
Furthermore, assume that the square image plane is 
transformed into a perspective image divided into pixels 
each having a width d and a height e. In expressions (4), 
(5), and ( 6 ) , the parameter W is changed in a range from 
W to -W on the units of W/d, and the parameter h is changed 
in a range from h to -h on the units of h/e, whereby 
coordinates of points on the square image plane are 
obtained. According to these obtained coordinates of the 
points on the square image plane, image data at points 
on the round- shape image formed on the imaging lens 2a 
which correspond to the points on the square image plane 
is transferred onto a perspective image. 
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Next, a horizontally rotational movement (pan 
movement) and a vertically rotational movement (tilt 
movement) in the perspective transformation are described. 
First, a case where point P as mentioned above is 
horizontally and rotationally moved (pan movement) is 
described. A coordinate of a point obtained after the 
horizontally rotational movement , point P " ( tx ' , ty ' , t z " ) , 
is represented as follows: 

tx" = (Rcos(|)+(h/2)sin(t))cos(e+Ae)-(W/2)sin(e+Ae) ... (7) 
ty' = (Rcos(|)+(h/2)sin(t))sin(e+Ae) + (W/2)cos(e+Ae) ... (8) 
tz' = Rsincj)- (h/2)cos(|) ...(9) 

where A6 is the movement angle. 

By combining expressions (7), (8), and (9) into 
expressions (2) and (3), the coordinate (X,Y) of a point 
on the round- shape image formed on the imaging lens 2a 
which corresponds to the point P' ( tx ' , ty ' , tz ' ) can be 
obtained. This applies to other points on the round- 
shape image. In expressions (7), (8), and (9), the 
parameter W is changed in a range from W to -W on the units 
of W/d, and the parameter h is changed in a range from 
h to -h on the units of h/e, whereby coordinates of points 
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on the square image plane are obtained. According to 
these obtained coordinates of the points on the square 
image plane, image data at points on the round- shape image 
formed on the imaging lens 2a which correspond to the 
point P' (tx* ,ty' ,tz' ) is transferred onto a perspective 
image, whereby a horizontally rotated image can be 
obtained. 

Next, a case where point P as mentioned above is 
vertically and rotationally moved (tilt movement) is 
described. A coordinate of a point obtained after the 
vertically rotational movement , point P" ( tx" , ty" , tz " ) , 
is represented as follows: 

tx" = (Rcos((})+A(t)) + (h/2)sin((t)+A(|>)xcose-(W/2)sine ... (10) 
ty" = ( Rcos{(|)+A(|)) + (h/2) sin ((J)+A(l))xsine+(W/2) cose ... (11) 
tz" = Rsin((()+A((>)-(h/2)cos{{))+A(|)) ...(12) 

where AcJ) denotes a moving angle. 

By combining expressions (10), (11), and (12) 
into expressions (2) and (3), the coordinate (X,Y) of a 
point on the round- shape image formed on the imaging 
lens 2a which corresponds to the point P" ( tx" , ty" , t z " ) 
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can be obtained. This applies to other points on the 
round- shape image. In expressions (10), (11), and (12), 
the parameter W is changed in a range from W to -W on the 
units of W/d, and the parameter h is changed in a range 
from h to -h on the units of h/e, whereby coordinates of 
points on the square image plane are obtained. According 
to these obtained coordinates of the points on the square 
image plane, image data at points on the round- shape image 
formed on the imaging lens 2a which correspond to the 
point P" ( tx" , ty" , tz" ) is transferred onto a perspective 
image, whereby a vertically rotated image can be obtained. 

Further, a zoom-in/zoom-out function for a 
perspective image is achieved by one parameter, the 
parameter R. In particular, the parameter R in 

expressions (4) through (12) is changed by a certain 
amount AR according to a certain key operation, whereby 
a zoomed-in/out image is generated directly from the 
round- shape input image formed on the imaging lens 2a. 

Furthermore, a transformation region 

determination function is achieved such that the range 
of a transformation region in a radius direction of the 
round-shape input image formed on the imaging lens 2a is 
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determined by a certain key operation during the 
transformation from the round- shape input image into a 
panoramic image. When the imaging section is in a 
transformation region determination mode, a 
transformation region can be determined by a certain key 
operation. In particular, a transformation region in the 
round-shape input image is defined by two circles, i.e., 
as shown in part (a) of Figure 4, an inner circle 
including the reference point O(ro,6o) whose radius is 
ro and an outer circle which corresponds to an upper side 
of the panoramic image 59 shown in part (o) of Figure 4. 
The maximum radius of the round- shape input image formed 
on the imaging lens 2a is rmax, and the minimum radius 
of an image of the imaging lens 2a which is formed on the 
imaging lens 2a is rmin. The radiuses of the above two 
circles which define the transformation region can be 
freely determined within the range from rmin to rmax by 
a certain key operation. 

Further still, the size of the perspective image 
(perspective transformation region) may be freely 
determined by changing the parameters W and h in the above 
expressions for the perspective transformation. 
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Next, a circuit configuration of a principal 
portion of the arithmetic/logic circuit 111 is described 
with reference to Figures 6 through 8. The 
arithmetic/logic circuit 111 includes at least one of a 
circuit for panoramic transformation (see Figure 6) and 
a circuit for perspective transformation (see Figures 7 
and 8 ) . 

Figure 6 shows an arithmetic/logic circuit 600 
for panoramic transformation. The arithmetic/logic 
circuit 600 includes two latch circuits 614 and 615, two 
multipliers 616 and 617, an adder 619, and a adder 618. 
Input terminals of the two latch circuits 614 and 615 are 
connected to a bus line 113. In response to control 
signals Ps and Pc, the two latch circuits 614 and 615 
respectively obtain values of sinB-j and cosBj through the 
bus line 113 from the lookup table (LUT) 110 of a 
trigonometric function (Figure 3) , and respectively hold 
these values. That is, the latch circuit (S-latch 
circuit) 614 holds the value of sin8j, and the latch 
circuit (C-latch circuit) 615 holds the value of cosO-) , 
Outputs of the latch circuits 614 and 615 are multiplied 
by ri by the multipliers 616 and 617, respectively. An 
output of the multiplier 617, (riXcosB-j), is added to Xo 
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by the adder 619, and the adder 619 outputs 
(X = Xo+ricosej) as X-coordinate data of a point on an 
input image. An output of the multiplier 616, (riXsinSj) , 
is added to Yo by the adder 618, and the adder 618 outputs 
(Y = Yo+riSinB-j) as Y-coordinate data of the point on an 
input image • These X-coordinate data and Y-coordinate 
data are transmitted through the bus line 113 to the 
output buffer memory 112 under the control of the CPU 109 
(Figure 3), Thus, in the image transformation 

section 104, under the control of the CPU 109 via the bus 
line 113, X- and Y-coordinate data is calculated based 
on image data stored in the input buffer memory 108, and 
the calculated X- and Y-coordinate data is stored at a 
corresponding coordinate position held in the output 
buffer memory 112. 

Next , an arithmetic/logic circuit for perspective 
transformation is described with reference to Figures 7 
and 8. The arithmetic/logic circuit for perspective 
transformation is formed by a first step of the 
arithmetic/logic circuit for perspective transformation 
(perspective transformation logic 1 of Figure 7) 
including four latch circuits 720-723, 

multipliers 724-727, 730-735, and 738, adders 729 
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and 736, and subtracters 728 and 737 and a second step 
of the arithmetic/logic circuit for perspective 
transformation (perspective transformation logic 2 of 
Figure 8) including multipliers 741-747, and 751, 
adders 740, 748, and 749, subtracters 739 and 752, a 
square rooter 750, and an inverter 753. 

In the perspective transformation logic 1 of 
Figure 7, input terminals of the four latch 
circuits 720-723 are connected to a bus line 113, In 
response to control signals 6s, 6c, (|)c, and , the latch 
circuits 720-723 respectively obtain values of sin8, cose, 
coscj), and sin(|) through the bus line 113 from the lookup 
table (LUT) 110 of a trigonometric function (Figure 3), 
and respectively hold these values. That is, the latch 
circuit 720 holds the value of sin6, the latch circuit 721 
holds the value of cos8, the latch circuit 722 holds the 
value of cos<t), and the latch circuit 723 holds the value 
of sincj). 

An output of the latch circuit 722 is input to the 
multipliers 733 and 734. An output of the latch 
circuit 723 is input to the multipliers 732 and 735. 
Outputs of the latch circuits 722 and 723 are multiplied 
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by R by the multipliers 733 and 735, respectively, and 
multiplied by h/2 by the multipliers 732 and 734, 
respectively. Outputs of the multipliers 732 and 733 are 
added by the adder 736. An output of the adder 736, r, 
is send to the multipliers 724 and 726. An output of the 
latch circuit 720 is input to the multipliers 724 and 725 . 
An output of the latch circuit 721 is input to the 
multipliers 726 and 727. Outputs of the latch 
circuits 720 and 721 are multiplied by r by the 
multipliers 724 and 726, respectively, and multiplied by 
W/2 by the multipliers 725 and 727, respectively. A 
difference between outputs of the multipliers 725 and 726 
is obtained by the subtracter 728 as tx, which is a 
x-coordinate of a point in a three-dimensional space: 

tx = (Rcos(|)+(h/2)sin(t))cos9-(W/2)sine 

Outputs of the multipliers 724 and 727 are added by the 
adder 729, and a result of the addition is output as ty, 
which is a y-coordinate of the point in a three- 
dimensional space : 

ty = (Rcos(l)+(h/2)sin(t))sin9+(W/2)cos9 
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The multipliers 730 and 731 generate tx^ and ty^ from tx 
and ty, respectively. 

A difference between outputs of the 
multipliers 734 and 735 is obtained by the subtracter 737 
as tz, which is a z-coordinate of a point in a three- 
dimensional space : 

tz = RsincJ)- (h/2 )cos(|) 

The multiplier 738 generates tz^ from tz* 

In the perspective transformation logic 2 shown 
in Figure 8, tx^, ty^ , and tz^ are added by the adders 748 
and 749. A result of the addition, i.e., an output of 
the adder 749 is input to a square rooter 750 so as to 
obtain a square root : 

sqrt ( tx^ + ty^ + tz^ ) = R(constant) 

An output of the square rooter 750 is multiplied by 2bc 
(constant) by the multiplier 751, and a result of the 
multiplication is output to the subtracter 752. In 
Figure 8, a, b, and c are mirror constants- 
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The adder 740 calculates a sum of and , and 
the subtracter 739 calculates a difference between b^ 
and c^ (b^-c^) . The multiplier 743 multiplies the sum of 
b^ and c^ by tz, and outputs a result of the multiplication 
to the subtracter 752. The subtracter 752 calculates a 
difference between the output of the multiplier 743 and 
the output of the multiplier 751. An output of the 
subtracter 752 is inverted by the inverter 753 into an 
inverse number, and this inverse number is transmitted 
to the multipliers 744 and 745. 

The difference between b^ and c^ (b^-c^) is 
multiplied by tx and ty by the multipliers 741 and 742, 
respectively. Outputs of the multipliers 741 and 742 
are respectively transmitted to the multipliers 744 
and 745 and multiplied by an output of the inverter 753. 
Furthermore, outputs of the multipliers 744 and 745 are 
transmitted to the multipliers 746 and 747 , respectively, 
and further multiplied by a constant F. The constant F 
is a focal distance of the imaging lens 2a (Figure 5). 
As a result, x- and y-coordinate data of a point on an 
input image are obtained as follows: 



- 40 - 



00R00703 



X = Fx( ( {b^-cMxtx/( (b^ + c^)xtz-2xbxcxsqrt(tx^ + ty^ + tz^) ) ) 
y = Fx( ( (b^-c^)xty/( (b^+c^)xtz-2xbxcxsqrt(tx^ + ty^+tz^) ) ) 

The X- and y-coordinate data is transmitted through the 
bus line 113 to the output buffer memory 112 under the 
control of the CPU 109 (Figure 3). Thus, in the image 
transformation section 104, under the control of the 
CPU 109 via the bus line 113, X- and Y-coordinate data 
is calculated based on image data stored in the input 
buffer memory 108, and the calculated X- and Y-coordinate 
data is stored at a corresponding coordinate position held 
in the output buffer memory 112. 

As described above, according to the present 
invention, the pan/tilt function for the panoramic image, 
the zoom-in/zoom-out function for the perspective image, 
and the transformation region determination function for 
the panoramic and perspective images are achieved by 
changing parameters (constants). Therefore, it is not 
necessary to add a circuit for these special purposes. 

As described above, all of the image 
transformations, e.g., the panoramic image 
transformation, the perspective transformation, and the 
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horizontally rotational movement and the vertically 
rotational movement (pan/tilt movement), are carried out 
only with linear operations. This is because the imaging 
section 102 is capable of acquiring an image which 
corresponds to an image seen from the focal point ® of 
the hyperboloidal mirror 54 (the optical system 101 is 
capable of central projection transformation) and, 
accordingly, the transformation from a polar coordinate 
to a rectangular coordinate can be carried out only with 
linear operations using the lookup table (LUT) 110 of a 
trigonometric function. Furthermore, the zoom- 

in/zoom-out function can be achieved with linear 
operations . 

In a conventional visual system, the above image 
data transformation is carried out using software . 
Accordingly, 20 or more steps are required in arithmetic 
processing for a single data transformation even in the 
case of using a hyperboloidal mirror. Therefore, the 
image transformation processing using the software 
requires a long processing time. Furthermore, it is 
further necessary to store intermediate results of the 
operations in a storage device and to read these 
intermediate results therefrom. On the other hand. 
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according to embodiment 1 of the present invention, the 
operations are carried out by the image transformation 
section (hardware) 104 without using software. Thus, it 
is possible to carry out the image data transformation 
within a delay time between an input and output of the 
hardware for operations (e.g., the image transformation 
section 104). Therefore, the present invention can be 
applied to the transformation for dynamic images, to which 
the conventional visual system cannot be applied. 
Furthermore, with the CPU 109 capable of parallel 
processing, faster processing speed is achieved. 
Further still, since the image transformation section 104 
is realized by hardware, the size of the entire 
omniazimuthal visual system can be reduced. 

(Embodiment 2) 

Embodiment 2 uses an optical system including a 
plurality of convex mirrors having a shape of a surface 
of revolution. Figure 9 shows an example of the optical 
system 101 capable of central projection transformation, 
including two convex mirrors having a shape of a surface 
of revolution. 

An example shown in Figure 9 uses first and second 
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paraboloidal mirrors 55a and 55b (a paraboloidal mirror 
is an example of a mirror having a shape of a surface of 
revolution). As shown in Figure 9, the first and second 
paraboloidal mirrors 55a and 55b are positioned such that 
the rotation axes of the first and second paraboloidal 
mirrors 55a and 55b are identical with the optical axis 
of an imaging lens included in the imaging section 102. 
The first paraboloidal mirror 55a, which is the closer 
of the mirrors 55a and 55b to the imaging section 102, 
has an opening 56. The imaging section 102 is positioned 
such that the first principal point of the imaging lens 
is located at a focal point of the second paraboloidal 
mirror 55b. With such a configuration, light which 
travels toward a focal point of the first paraboloidal 
mirror 55a is reflected by the first paraboloidal 
mirror 55a, and the light reflected by the first 
paraboloidal mirror 55a is further reflected by an 
internal face of the second paraboloidal mirror 55b so 
as to reach the first principal point of the imaging lens. 
In the optical system having such a structure, an image 
obtained by the imaging section corresponds to an image 
seen from a focal point of the first paraboloidal 
mirror 55a. 
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Such an optical system is described in detail in 
Japanese Laid-Open Publication No. 11-331654. An 
omniazimuthal visual system including an image 
transformation system of the present invention can use 
this optical system. 

As described hereinabove in detail, according to 
the present invention, an image transformation section 
is formed by hardware including an arithmetic/logic 
circuit. Therefore, the processing speed for image 
transformation is faster than that of a conventional 
system which carries out image transformation using 
software. Thus, the system of the present invention has 
sufficient performance which can provide a processing 
speed required for dynamic images even when the number 
of pixels to be processed is increased. Furthermore, 
since the image transformation section is realized by 
hardware, the size of the entire omniazimuthal visual 
system can be reduced. 

Furthermore, according to the present invention, 
the zoom-in/zoom-out processing or the pan/tilt 
processing are carried out based on image data obtained 
by the imaging section, and then, image data corresponding 
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to an image which has undergone such processing is 
transformed into a panoramic image or a perspective image. 
With such a processing procedure, the number of processing 
steps and the use of a buffer memory can be reduced and, 
accordingly, the processing speed can be further improved, 
and the size of the system can be further reduced. Further 
still, when a CPU having a parallel operation function 
is used, a still faster processing speed for image 
transformation can be realized. 

Various other modifications will be apparent to 
and can be readily made by those skilled in the art without 
departing from the scope and spirit of this invention. 
Accordingly, it is not intended that the scope of the 
claims appended hereto be limited to the description as 
set forth herein, but rather that the claims be broadly 
construed. 



